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ABSTRACT 

This publication informs policymakers of the breadth 
of the elements of quality and suggests an integrated framework for 
assessing quality to policymakers and vocational education 
evaluators. Chapter 1 discusses the development of a proposed 
framework identifying the elements of vocational education quality. 
The framework is composed of three critical components: context, 
educational experience, and vocational outcomes. Chapter 2 introduces 
and defines the elements of quality related to educational outcomes. 
These elements are categorized under two general types of outcomes: 
individual and societal, with a subset of societal outcomes specific 
to employers. Chapter 3 parallels chapter 2 in structure but focuses 
on the elements of quality related to the educational experience. It 
discusses the school context and school process, and their 
components, which define the boundaries of the school context and 
school process domains. Chapter 4 presents some issues and guidelines 
regarding the development of quality indicators for the elements 
within the proposed model. It defines quality, describes approaches 
to assessing quality, and then suggests a process for identifying and 
prioritizing subelements within the model. References are appended. 
(YLB) 
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FOREWORD 



Traditionally, vocational education program quality has been defined (1) in terms of the output 
of the program (e.g., training-related jobs obtained by former students, satisfied employers) or 
(2) as the process of the program (e.').. time spent on learning tasks, skill content addressed). Both 
of these are logically related to quality, but neither one by itself provides an explanation of quality 
that is sufficient to influence policy toward quality improvement. 

This publication provides a discussion of the elements of quality in vocational education as 
viewed in the context of formal education. Reviews of these elements by knowledgeable practi- 
tioners and recommendations related to the development of quality indicators are also included. It 
is an attempt to conceptualize a comprehensive framework for evaluating vocational education 
quality. The report is intended to inform policymakers of the breadth of the elements of quality and 
to suggest an integrated framework for assessing quality to vocational education evaluation 
theoreticians. 

This project was conducted in the Evaluation and Policy Division of the National Center for 
Research in Vocational Education under the direction of N. L. McCaslin, Associate Director. Many 
people made significant contributions in the course of its completion. We wjsh to thank for their 
participation in our field discussions Dr. Nellie Dry, Dr. George Mehallis, Charlotte Hayes, and 
David Hall, associated with Broward Community College in Ft. Lauderdale, Florida; John Ryan. 
Ron Smith, Joe Fabing, and Billy Keith Butt, associated with Tri-County Joint Vocational School in 
Nelsonville. Ohio; Dr. Dwight Davis, Marvin Bausman, Dan Sczygelski, Larry Post, and Cindy 
Pogachnick, associated with North Central Technical Institute in Wausau, Wisconsin; Dr. Jim 
Frasier. Nelson Whitemen, Harold Darling, Harold Pearson, and Beth Allen, associated with Patter- 
son Cooperative High School in Dayton. Ohio; and Ray Bertleson. John Miller, Marilyn Gordon, 
and Matt Medlock. associated with Westerville South High School in Westerville. Ohio. 

We also acknowledge the contributions of National Center staff members Roy Butler. Ida 
Halasz. N. L. McCaslin, Allen Wiant, and Jane Reisman and the field work of Cathy Warmbrod and 
Jim Hamilton, which was instrumental in the development o our model. 

David L Passmore. professor of vocational education, The Pennsylvania State University. 
George Quarles. former director of vocational education for New York City, and Robert H. 
McCabe. president. Miami-Dade Community College, provided helpful criticisms and suggestions 
related to the overall context and outcomes of our conceptual framework. 

We wish also to thank the project staff— Paul Campbell, project director, Jim Hamilton, Cathy 
Warmbrod, and Phyllis Panzano for their work in preparing the publication. And finally, we thank 
Jeanne Desy for her comments related to the organization of the report, Mary Beth Dauner for her 
effort, cooperation, and patience in typing the manuscript with its many revisions, and Janet 
Kiplinger, under whose supervision the publication was edited. 



\ 

Robert E. Taylor 
Executive Director 
National Center for Research 
in Vocational Education 
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EXECUTIVE SUMMARY 



The elements of quality proposed in this publication have. b*en derived from a diverse body of 
literature and experience. We have attempted to incorporate various findings related to attributes 
of quality education and programs into a general yet comprehensive conceptual framework in 
order to broaden our understanding of the elements of quality vocational education. It will con- 
tribute to an increased awareness of the major components influencing the quality of education 
This publication is intended to inform policymakers of the breadth of the elements of quality and 
to suggest an integrated framework for assessing quality to policymakers and vocational educa- 
tion evaluators. 

For purposes of this effort vocational education was defined as school-related activities that 
occur in secondary schools and 1- and 2-year programs in postsecondary schools; vocational 
education was considered to be a subset of general, formal education, which in turn is a subset of 
education coming from a variety of life experiences. 

Our framework for the elements of quality grew out of a multiphased process that included 
reviews of related literature, expert input, and key informant interviews. The literature provided 
preliminary structure to the framework that was further refined and defined by expert opinion and 
key informant commentaries. 

Some vocational education has been directed by law to be evaluated in terms that include job 
placement in training-related employment employer satisfaction, and quality and availability of 
instructional offerings. This effort, however, initially explored a variety of desirable outcomes that 
may help to define high quality vocational education programs and subsequently identified the 
components of the educational environment that may support attainment of those outcomes. The 
outcomes were therefore considered as goals and the structures and processes within the educa- 
tional environment were perceived as means for achieving the goals. The resulting framework for 
the elements of vocational education quality (figure 1) was conceptualized as a complex environ- 
ment converging on the educational experience, driven by desired individual and societal out- 
comes, and influenced by environmental inputs such as community setting and labor market 
conditions. 

The Framework: Elements of Quality 
Vocational Education 

Elements of Quality Outcome 

Two general types of outcomes are included in our framework: individual and societal with a 
subset of societal outcomes specific to employers. Four elements of quality relating to individual 
outcomes were identified and defined. Competency development encompasses the development 
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of work-related skills (both occupational and job specific), basic skills such as verbal and mathe- 
matical, and general problem-solving ability. Positive interpersonal interaction involves the ability 
of people to have positive and mutually supportive relations with others in the home, the work- 
place, and the community. A sense of efficacy is an attitude of confidence in one's ability to utilize 
skills to achieve goals. The ability to manage a large array of tasks without having to depend upon 
someone else is designated as a sense of independence. These are the four general elements of 
quality individual outcomes included within our conceptual framework. 

Social stability and employer needs are the primary elements of quality within the societal 
outcome portion of our framework. Social stability does not imply social stagnation; rather, it 
means gradual, orderly change. Provision of essential services, quality of work products, resources 
for self-support, and societal improvement are the subelements that support social stability. 

We also examined a subset of societal outcomes specific to employers. To be worth the 
investment from the employer's point of view, vocational education should contribute to available 
skills, reduced training time, and high productivity.Jhese are the three elements of quality related 
to employers needs. Our definition of quality vocational education outcomes includes, but is not 
limited to. employment-related skills. Rather, vocational education of a positive quality must also 
enhance the development of desirable or even, essential skills or dispositions that are not directly 
employment related. 

Elements of Quality within 
the Educational Experience 

Taking a systems perspective, the processes and structures that define the elements of quality 
within the vocational education experience should relate to intended individual and societal out- 
comes. Our model divides the educational experience into two domains, school context and 
school process. School context deals with both physical and organizational aspects of the school 
unit. Personnel, facilities, equipment, policies, and formal organizational structure all fall within 
school context. The elements of quality within school context that are formally included in our * 
model are school setting, classroom setting, teacher skills, teacher characteristics, and peer group 
characteristics. Figuratively speaking, school context provides the setting for school process, 
which provides the "action." Estimations of quality of school process would consider such things 
as course content; course-specific equipment; interaction between teacher, student, and materials; 
and level of formal student involvement in the vocational education program. The elements of qual- 
ity formally included within school process are feacn^r. student, and material interaction, instruc- 
tional content and material; and student utilization of instructional process. 

These elements of quality within the educational experience should relate to intermediate 
objectives that, in turn, should support attainment of desired outcomes; the linkage should be sys- 
tematic between quality of experience and quality of outcome in order to maintain and replicate 
effective programs and make modifications as necessary. 



Assessing the Quality of Vocational Education 

The elements of quality within our framework are generalizable across school unit and pro- 
gram. However, the specific indicators of quality associated with them may vary across school and 
program and are likely to vary overtime. 



x 



Considerable effort has been expended in developing evaluative strategies to facility, e the 
assessment of quality. Evaluation models should combine structural, process, and outc ..ne 
approaches in order to assess the range of elements of quality within our framework. 

Specific indicators of quality need to be developed at the individual school unit level. It is not 
practically possible to tabulate exhaustively all the indicators related to the elements of our model, 
nor is it.financiaily practical to support an evaluation system capable of gathering and analyzing 
such data in a complete form. A process for identifying and prioritizing subeiements within the 
model is suggested as a practical starting point for developing indicators. A combination of quan- 
titative and qualitative methods is suited to a comprehensive evaluation of quality, and both 
normative and empirical approaches should be drawn upon for developing indicators. 

In summary, this discussion presents a justification and set of procedures that will permit an 
integrated consideration of the elements of vocational education quality. We have advocated con- 
sidering a broad set of elements of quality rather than a narrowly defined or segregated set. so that 
the evaluation of vocational education reflects its contribution to a set of living expe/iences in 
addition to employment. 
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CHAPTER 1 



INTRODUCTION 
Some Preliminary Considerations 



What Is This Publication About? 

When determinin^bhether or not a pro- 
gram of instruction is producing the desired 
outcomes, it is first necessary to decide what 
those outcomes should be. Vocational educa- 
tion has been directed by law to be evaluated 
in terms of job placement in training-related 
employment and employer satisfaction. 
Although both of these outcomes are reason- 
able as well as legally required, there is a 
large body of opinion that extends the 
desired outcomes beyond that limited set. 
This publication advocates such an expan- 
sion for the purpose of evaluation, explores a 
variety of potentially desirable outcomes that 
may help to define a high-quality vocation! 
education program and identifies those 
components of the educational experience 
that may support attainment of those 
outcomes. 

The elements of vocational education 
quality may be found in two places— the out- 
comes of the educational experience and the 
educational experience itself. In this discus- 
sion we present a tentative set of outcome 
goals, with some description of their nature 
and justification for their selection. The justi- 
fication relies upon normative premises, 
empirical and theoretical evidence, and, in 
some cases, intuitive logic. Because the pro- 
cesses and structures that presumably define 
the educational experience should relate to 
the intended outcomes, the outcomes are 
considered first. A similar consideration of 
the processes will be the topic of a subse- 
quent chi pter. 

We are attempting to conceptualize a 
generai yet comprehensive framework for 
assessing vocational education quality. It will 
contribute to an increased awareness of the 



breadth of the components that influence the , 
quality of education. 

What In Vocational Education? 

Before proceeding with a search for ele- 
ments of quality in vocational education, 
some fences must be erected around the 
varied terrain of ideas and activities to which 
the label 'Vocational education" is attached. 
From a narrow point of view, vocational edu- 
cation is considered by some as only those 
school-related activities that are the recip- 
ients of Federal funds under the vocational 
education acts beginning with the Smith- 
Hughes Act in 1917. On the other hand, it 
may be considered as any school-related 
activity, through graduate and professional 
schools, that provides skills for employment 
in a specific field— from machining to medi- 
cine, or from cooking tp counseling. For pur- 
poses of this inquiry, we have elected to limit 
consideration to those school-related activi- 
ties that take place in secondary schools and 
in 1- or 2-year programs in postsecondary 
schools. These school activities that we clas- 
sify as vocational education include— but as 
we intend to show subsequently— are not 
limited to programs that produce salable 
skills specific to general classes of occupa- 
tion. They may or may not be supported by 
Federal funding. 

Where Does Vocational Education Fit? 

We consider vocational education to be a 
subset of general, formal education, which, in 
turn, is a subset of education coming from a 
wide variety of life experiences. At the 
secondary level, it is part of a universal and 
obligatory educational experience. It is uni- 
versal in most Western democracies because 



provision is made in the laws of these States 
to provide it for nearly everyone. The most 
usual exception occurs fvhere some 
extremely severe mental or physical problem 
renders formal education impractical. Educa- 
tion is an obligation because most of these ' 
States require some level of compulsory * 
attendance. 

At the postsecondary level, the obliga- 
tory feature of education is completely 
relaxed. The universal quality is also much 
less apparent, although the general availabil- 
ity of public funds to support postsecondary 
institutions, at least in the United States, 
reflects a commitment to broaden the access 
of individuals to continued education. 



What Should Be»the Outcome of General, . 
Formal Education? 

To understand where vocational educa- 
tion fits within this scope of general, formal 
education, it is necessary to examine phat 
the outcomes of general, formal education 
are expected to be. Frankena (1974) de- 
scribes these outcomes as the formation of 
what he calls "dispositions" that include abili- 
ties; habits, beliefs, skills, and bodies of . 
knowledge. , 

It is beyond the scope of this discussion 
to elaborate upon the basic normative prem- 
ises undetlying the selection of the disposi- 
tions that seem to be most universally valued 
in American society. Rather, those that have 
broad representation in the literature, in the 
media, and in practice suggest what appears 
to be the society's underlying values^ If we 
consider both practice and commentaries, 
the currently existing values, as well as those 
desired, have a chance to emerge. 

There seems to be general agreement 
that speaking, reading, and writing the major- 
ity language, as well as handling basic quan- 
titative problem solvi ig, are fundamental 
elements of the dispositions. At least some 
level of comprehension of the procedures of 
science and the political organization of the 



Nation. State, and community are also 
generally accepted. Some National reports 
(e.g.. by the National Commission on Excel- 
lence. The Twentieth Century Fund) have 
additionally emphasized such other disposi- 
tions as computer literacy and foreign 
languages. - 

That acquisition of the foregoing disposi- 
tions (e.g.. language, math) is necessary for a 
• good and productive life seems to be sup- 
ported by the observable, normative prem- 
ises. There is no general agreement about 
how all of these basic skills should be pro- 
vided, but the presence of some elements of 
language, government, science^ and math 
curriculum in virtually all high schools 
underscores the importance attached to for- 
mal study of these skills by American society 

All societies are made up of individuals 
and of institutions that, at the time of their 
founding, were intended to provide some 
lorm of service to individuals. (This purpose 
may have been lost to self-perpetuation as 
the institutions developed.) American 
society, in Which the form of vocational edu- 
cation being examined is set, has a long 
tradition of emphasis upon the individual. 
Illustrative are the writings of Thomas 
Jefferson on the need for an educated citi- 
zenry. Abraham Lincoln's eloquent reference 
to "government for the people, by the people, 
and of the people" in the Gettysburg address, 
and John Dewey's concern for the role of the 
individual, expressed in the following quote: 

Such societies were found to be demo- 
cratic in quality, because of the greater 
freedom allowed the constituent 
members, and the conscious need of 
securing in individuals a consciously 
socialized interest, instead of trusting 
mainly to the force of customs operating 
under the control of a superior class The 
sort of education appropriate to the 
development of a democratic community 
was then ^/plicitly taken as the criterion 
of the further, more detailed analysis of 
education. (Dewey 1916. p. 322) 
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In this society, education is focused upon 
individuals and seems to have two basic 
g 0a i s _to help individuals maximize their 
well-being in a broad sense and to provide a 
stability in the society through ensuring the 
well-being of its members collectively. 

An argument may also be made that 
work is necessary for the good and produc- 
tive life. As John Gardner (1961) puts it, "The 
"best-kept secret in America today is that 
people would rather work hard for something 
they believe in than enjoy a pampered idle- 
ness." Although the context of Gardner's 
remark is the need to invest human energy in 
improving society in general, it applies to 
vocational education, which can and does 
address directly a specific inclination to work 
that serves both societal and individual ends. 
One can easily recall numerous expressions 
of the desire to work ranging from expecta- 
tions to continue working expressed by lot- 
tery winners, through the search for part-time 
jobs by retired persons, to the pride with 
which workers display tangible results of 
their efforts. Vocational education provides 
Specific training that enables people to work, 
and thus addresses the individual's need 
represented by these expressions as well as 
society's need for the outcomes of an indi- 
vidual's work. v 

Vocational education cannot be pursued, 
however, at the expense of some acceptable 
level of the broad basic dispositions that 
were detailed earlier. That these are generally 
valued is evidenced by their prevalence and 
also by their support by National sounding 
boards. To quote Dewey (1916) again: 

Put in concrete terms, there is danger 
that vocational education will be inter- 
preted in theory and practice as trade 
education: as a means of securing tech- 
nical efficiency in specialized future 
pursuits. 

Education would then become an 
instrument of perpetuating unchanged 
the existing industrial order of society, 
instead of operating as a means of its 



transformation. The desired transforma- 
tion is not difficult to define in a formal 
way. It signifies a society in which every 
person shall be occupied in something 
which makes the lives of others better 
worth living, (p. 316) 

The National Commission on Secondary 
Vocational Education seconds Dewey's 
concern— "Vocational education should not 
dilute the student's general education nor 
disenfranchise certain groups through selec- 
tive enrollment" (1984, p. 11). Therefore, 
vocational education as we have defined it is 
part of the broader concept of education with 
its general goal of individual and societal 
\ well-being. It shares with general, formal 
education the dispositions that are presumed 
to lead toward these two aspects of 
well-being. 

Conceptual Framework 

Before going further, it may be useful to 
consider the overall setting in which voca- 
tional education operates. Figure 1 has been 
prepared to illustrate one schematic of the 
conceptual framework. Briefly, figure 1 dis- 
plays three critical components of the overall 
setting. The left-hand panel displays the 
major dimensions of the context. These are 
conditions that exist independently of, and in 
most cases, prior to, participation in voca- 
tional education. Although the figure does 
not make it explicit, the general context con- 
tinues to operate beyond formal education 
and influences independently the effective- 
ness of the outcomes in accomplishing the 
goals that underlie a cohception of quality. 
The central panel displays the major dimen- 
sions of the vocational education setting. 
Three categories comprise this panel. They 
are the school-specific context, the teaching- 
learning process, and the formal utilization of 
that process by the student. The right-hand 
panel displays a set of outcomes. These are 
divided into two major categories, individual 
and societal. ^ N 

The outcomes panel is discussed first in 
this report. Discussion of the school setting, 
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CONTEXT 



EDUCATIONAL 
EXPERIENCE 



VOCATIONAL 
OUTCOMES 



Family background 



Personal Characteristics 




Labor Market Conditions — 



School Context 



School Setting 
Classroom Setting 

Teacher Skills 
Teacher Characteristics 

Peer Group Characteristics 



1 



Student Utilization 
of Instructional Process 
(Patterns of Participation) 



I 



Teacher, Student, and 
Material Interaction 



I 



Instructional Content 
and Equipment 



School Process 



Individual 

Competency development 

Basic skills 

Information skills 

Problem-solving skills 

Employment skills 

Job-specific skills 

Career awareness 
Positive interpersonal interactions 
Sense of efficacy 1 
Sense of independence 

Enrichment 

Self helping skills 



Societal 

Social stability 
Essential services 
Quality of work products 
Resources for self-support 
Societal improvement 

Employer needs 
Available skills 
Reduced training time 
High productivity 



Figure 1 . Framework identifying the elements of vocational education quality 



teaching-learning process, and student utili- 
zation will follow. The overall approach can 
best be understood, however, if the concep- 
tual framework is kept in mind. 

The approach taken here assumes a 
functional role for education, rather than a 
sorting or conditioning role (e.g., Bowles and 
Gintis 1976) or a role of securing a privileged 
position in the job queue (Thurow 1975). 
Human capital theory might be regarded as 
the basis for this approach, but we take the 
position that rational decisions in investing 
for desired returns are not sufficient to cap- 
ture the overall value and meaning of human 
existence. Thus, although our discussion will 
be in many respects consistent with human 
capital conceptions, we do not limit ourselves 
to expressing the values we have accepted 
nor the outcomes we discern as having nor- 
mative value, within the confines of that theo- 
retical perspective. In other words, we accept 
an altruism that values human existence in its 
own right, independent of economic 
interpretations. 

We also recognize that the institutions 
delivering formal education reflect the con- 
flicting forces and differences of opinion that 
permeate the society. They well may serve as 
a sorting mechanism, in part a function of 
their time-bound nature and the imperfect 
professionals who structure the experiences 
for students within them. The results of 
V acquiring the dispositions may be interpreted 
\by employers as proxies for the ability to 
Item new skills as yet unknown. As a conse- 
quence, noncredentialed workers may be 
^displaced or not hired, even though they 
possess the needed skills or ability to learn. 
These conditions reflect the imperfections of 
the enterprise that we seek to understand in 
order to improve. We therefore consider the 
outcomes as goals and the processes as 
ways to move closer to the goals. 



Modeling the Elements of 
Vocational Education Quality 

The proposed framework for the ele- 
ments of vocational education quality was 



the result of a multiphased process A review 
of the literature, expert judgment, and key 
informant interviews provided input to the 
modeling process. 

The literature on effective school n 
research and related subjects was the initial 
source considered in developing the model. 
Although the literature directly relating to 
and identifying the elements of vocational 
education quality was quite segmented, ref- 
erences were discovered that addressed vir- 
tually all the major components of our 
framework. The content of the references 
ranged from general observations and com- 
ments about effective programs to sugges- 
tions for specific indicators of quality. 
Related references were subsequently 
grouped and the resulting significant clusters 
were incorporated as model components or 
elements. 

To illustrate briefly the method by which 
references were utilized in model building, 
consider the following examples. Miller 
(1984) and McKinney et al. (1981 ) comment 
that effective schools are characterized by 
total commitment of the school unit to quality 
programs. This reference could be generally 
classified as an organizational characteristic 
of school setting. More specifically, it could 
be considered as an example of a general 
policy statement that is common to quality 
programs. References were also found relat- 
ing to organizational structure (Westbrook 
1982) and strategy (Benson 1982). This type 
of information was subsequently utilized to 
build a preliminary geqpral framework of 
school setting that was stftjctured as follows: 

• Organizational Aspects 
—structure 
—policy 
—strategy 

As more information was gathered, a 
physical aspect of school setting was also 
identified, structured, and added to the 
model. The process was followed and even- 
tually resulted in a preliminary, systematic 
framework identifying the elements of voca- 
tional education quality. 



Experts in the field were involved in 
model definition througn normative group 
processes, modified Deiphi approaches, and 
individual review. Expert input was utilized to 
cross-validate components suggested by the 
literature, to suggest additional components 
to our model, and to develop more detailed 
definitions of components. 

The final source of information was key 
informant interviews. Nine administrators, six 
instructors/department heads, and six pro- 
gram graduates representing secondary and . 
postsecondary vocational education pro- 
grams participated in an unstructured inter- 
view process. Interviews were conducted as 
open discussions on the topic of "Quality 
Vocational Education/' The role of the inter- 
viewers was to keep the discussion focused 
on the general topic and to provide prompts 
or suggestions for related discussion. 

In observance of Federal regulations 
regarding over-collection of information, the 
sample size was intentionally kept small 
Even though they were not technically repre- 
sentative of the population of interest, the 
interviews were valuable to the model- 
formulating process. Interviews were content 
analyzed and informally used to cross- 
validate the basic model suggested by expert 



judgment and the literature. Interview infor- 
mation also aided in defining the 
components and elements of quality and in 
providing examples of potential indicators of 
quality. 

In summary, our model was developed 
through a synthesis of the information 
gathered from experts, practitioners, and 
related literature. The model is proposed as 
an attempt to define in an integrated fashion 
the people, processes, and structures that 
define the elements of vocational education 
quality. 

Organization of This Publication 

The remainder of this publication is 
divided into three chapters. Chapter 2 
introduces and defines the elements of qual- 
ity related to educational outcomes that were 
identified through the previously described 
model-building process. Chapter 3 parallels 
chapter 2 in structure, but focuses on the 
elements of quality related to the educational 
experience. Finally, chapter 4 presents some 
issues and guidelines regarding the devel- 
opment of quality indicators for the elements 
within the proposed model. 
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CHAPTER 2 
EDUCATIONAL OUTCOMES OR GOALS 



Two kinds of outcomes or goals are sug- 
gested in figure 1: individual and societal, 
with a subset of societal outcomes specific to 
employers. They are not presented as 
exhaustive, but were selected to represent 
the key elements deemed necessary for a 
productive and satisfying !ife. 



Individual Outcomes 

Within the parameters of individual out* 
comes are several components. The first of 
these is called competency development. 
Aspects of this outcome include the devel 
opment of work-related skills, both occupa- 
tional and job specific. They include the 
development of the basic skills suggested 
earlier in this paper as being part of the 
general right of individuals and obligation of 
education. They also include the develop- 
ment of general problem-solving ability. 

It should be understood that, as stated 
earlier, the development of the more recog- 
nizable vocational skills is not confined to 
formal education, either secondary or post- 
secondary; rather, both of these levels pro- 
vide avenues for their development. The 
processes (methods) currently employed to 
develop the necessary dispositions in the 
basic skills, again using Frankena's terms 
(1974), are primarily those of formal educa- 
tion. This premise is supported by law, by 
published opinion, and by traditional 
practice. 

Examples of the occupational and job- 
specific skills are such things as drafting, 
keyboarding, engine troubleshooting, pro- 
gramming, and crop planning. There are also 
more general, occupation- and job-related 
skills that include knowledge of the world of 



work, job search strategies, interview pro- 
tocol, and workconventions (arriving on 
time, following work rules). In most work 
situQtions, some ability to communicate 
using either spoken or written language is 
necessary. In many, the solution of quantita- 
tive problems is another significant element. 
The general ability to make appropriate 
judgments about the consequences of one's 
actions has major importance. Also., a recog- 
nition of fundamental relationships that are 
the stuff of science plays a part, whereas 
specific understanding of certain chemical, 
physical, biological, and economic laws fig- 
uresln the world of work in many instances. 
For example, failure to understand the role of 
soil nutrients leaves the development of a 
crop to happenstance. Or, the likelihood of a 
business failure is increased by not under- 
standing that when costs exceed revenues, 
the enterprise is bound to fail when its avail- 
able capital is depleted to a critical level. 
Competency development is judged to be a 
key component because it provides the indi- 
vidual with a currency to exchange for 
needed goods and services, the necessary 
dispositions to accomplish self-services ■ 
independently of other people, and the ability 
to add to the context of life in a positive way 
through contributions to society. 

Another individual competency with 
increasingly important potential might be 
identified as "information skills.' 1 * As the 
society changes from an industrial base to an 
information base, a great variety of new 
sources and kinds of information become 
important— and as jobs change, these altera- 
tions frequently require accessing new 
information. The need for this skill is implied 
within many of the basic skills, but it is 
worthy of emphasis as a separate and distinct 
skill because of its importance. It includes 



•This conception was contributed by one of our reviewers. Robert McCabe. President. Miami-Dade Community College 
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finding, evaluating, applying, and communi- 
cating information. It will apply not only to 
stability and growth within a vocation, but 
also to other aspects of living as well. 

The second component, positive inter- 
personal interaction, permeates not only the 
work role of individuals but also the total life 
space. Here individual fulfillment of nearly all 
people depends on having positive and mu- 
tually supportive relations with key people 
whose lives mesh with theirs in the work- 
place, the home, and the larger community. 
To do this, they need to be able to communi- 
cate, to be sensitive to the impact that they 
have on their colleagues, and to develop and 
maintain a sense of shared responsibility. 

A wide range of interpersonal interac- 
tions might be labeled positive. At the one 
extreme are those people, some employers 
among (hem, who want docile response to 
requests and directives. At the other are 
those who demand individual assertiveness 
without regard for the consequences to oth- 
ers. We define positive, interpersonal actions 
as being neither of these extremes but rather 
a set of behaviors involving other people. 
These behaviors are adjusted to the circum- 
stances, but the individuals do not accept 
indefinitely repressive situations: they are 
responsive to the needs o f others, but not 
insensitive to their own fulfillment. This point 
of view is analogous to Sternberg's (1984) 
— — contextuafsutotheory of adaptation, selec- 
tion, and shaping. According to Sternberg's 
theory, individuals first try to adapt to their 
environments. If this is impossible or undesir- 
able, they try to select a new environment. 
Should that not work out, they try to adapt 
the environment to make it a better fit. Posi- 
tive interpersonal relations involve activities 
of this kind We do not assume, however, the 
order implied here. We do assume that the 
disposition or behaviors required can be 
developed, rather than being inherent. 

A summary of research consistent with a 
definition of this sort is also found in 
Tennyson et al. (1980;. These authors con- 
sider both the work situation and other social 



settings. They identify the functionality of 
positive interpersonal relations, which they 
call "mature," in a variety of settings. In partic- 
ular, they imply that better problem solving 
will result *rom those mature relationships 
because less effort will be spent on defensive 
reactions and more on eliciting information 
and developing solutions. These constructive 
actions will lead to reduced conflict and less 
occurrence of defensive behavior. 

A third individual outcome that we postu- 
late is a sense of efficacy. Stated simply, it 
implies that individuals not only have some 
skills, but also sense their ability to use them 
to accomplish something. They expect to be 
able to get a job, to accomplish tasks deemed 
to be worthwhile, and to exercise a reason- 
able degree of control over their immediate 
personal environment. 

Beginning with the postulations of Kurt 
Lewi^ that the likelihood of undertaking a 
task ik related to an individual's perception of 
the likelihood of success, the role of a sense 
of efficacy in motivation has been detailed by 
numerous researchers. (For example, see 
Kelly 1967; Wong and Weiner 1981; Taylor 
1983.) \ 

The theories developed by these 
researchers and the evidence they have 
examined strongly suggest the importance of 
a sense of efficacy. In contrast, the concept 
of learned helplessness (Abramson, 
Seligman, and Teasdale 1978) and of its con- 
sequences illustrates the debilitating effect of 
the opposite pole of the efficacy continuum. 
Also, there is empirical evidence that a sense 
of efficacy is associated with certain aspects 
of job satisfaction. For example, a recent 
study of job satisfaction, utilizing the 
National Longitudinal Survey Youth data 
(Campbell et al. 1982) found a consistent 
association between four different forms of 
job satisfaction and scores on a scale that 
included items such as Tm able to do things 
as well as most other people." These exam- 
ples of theoretical and empirical evidence 
add to a normative and humanistic concern 
for the qualities of individual satisfaction to 
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support the inclusion of a sense of efficacy 
as a major individual outcome of formal 
education. 

The fourth major outcome for individuals 
includes to some degree the preceding three. 
It is designated as a sense of independence. 
It means that the individual can manage a 
large array of ordinary tasks without having 
to depend upon someone else. It means that 
a man can provide a balanced meal when 
needed and that a woman is not helpless 
when confronted with a flat tire. It means that 
the individual chooses among alternatives 
rather than being channeled along a single 
pathway of life episodes. 

A sense of independence has a strong 
normative background demonstrated in such 
diverse documents as the Declaration of 
Independence and the literature on therapeu- 
tic counseling. Its relationship to specific 
work life as well as its role in the general 
quality of life is suggested by a variety of evi- 
dence. It is reportedly related to job satisfac- 
tion among adults. For example, in a recent 
study, Hills and Crowley (1983) found that a 
sense of independence, operationally defined 
as a sense of autonomy on the job, was sig- 
nificantly related to a scale score of job satis- 
faction. This result is a further verification of 
Robert Blauner's classic study, Alienation 
and Freedom (1964). Katzell (1979) also pro- 
vides an analysis of the growing awareness 
of the legitimacy of this sense among the 
current work force. Although these 
relationships do not establish a sense of 
independence as a goal, they document its 
pervasiveness as a human value, which, in 
our judgment, is the ultimate criterion for 
justifying it as a goal. (See also Taylor [1974] 
for a discussion of the lojical and philo- 
sophical basis for this contention.) 

These four individual outcomes rest 
upon a class of intermediate outcomes that 
are not so much ends in themselves but are 
necessary in some form in order to attain the 
more f undam *ntal results. Examples are the 
development of self-helping skills (tire chang 
ing, cooking, keyboarding), information 



about alternatives (career awareness), and 
the broadening of each individual s repertoire 
of knowledge (enrichment). Vocational edu- 
cation functions with respect to these out- 
comes in varied ways. In some cases, it may 
be the sole, school-based source of oppor- 
tunity to achieve the outcome (cooking). In 
others, achieving the outcome may be shared 
with alternative curricula (general, academic) 
in the formal school setting. 

Societal Outcomes 

The societal outcomes are five in 
number. In order of generality, social stability 
is probably highest. It is necessary for the 
other intermediate outcomes to be possible, 
but it also depends, at least in part, upon 
their attainment. By social stability we do not 
mean social stagnation. Rather, We mean 
gradual, orderly, and nonviolent change, with 
expectations within the society being rea- 
sonably predictable, at least for a long 
enough period that changes may be planned, 
rather than being unpredictable shocks. 
Neither education in general, nor certainly 
vocational education in particular, can 
accomplish this outcome alone, but the expe- 
riences that occur in vocational education 
should meet the test of a positive contribu- 
tion to social stability. 

The remaining four societal outcomes 
are much more specific. They represent prod- 
ucts that can be addressed quite directly by 
vocational education. 

Providing essential services to the 
society is an obvious need. Trained people in 
health care, information processing, and the 
highly skilled components of durable manu- 
facturing are needed for a smoothly function- 
ing, modern society. It is recognized that not 
all essential services are provided through 
the labor market or through paid work. The 
quality of a society also rests heavily upon 
those services that are provided in the home 
and by volunteers in various roles. 
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Although it is true that services viewed as 
essential by the majority of the society are 
probably available through some form of 
enterprise that becomes part of the labor 
market, not all services are so provided. 
Many are available both on a paid basis and 
on a donated basis. Numerous examples 
exist of the latter, all of which contribute to 
the quality of life in a society but do not 
require an exchange of money, goods, or 
services. Volunteers often provide some form 
of nursing care to sick patients, technical 
advice to amateurs on home repair, free 
assistance on car repairs, free garden pro- 
duce, snow shoveling, attentive concern to 
people in trouble, family management, par- 
enting, teaching (e.g, church school informal 
music lessons), advice on nutrition, politics, 
and ways to cope with a complex society. 
Whether government policies and profes- 
sionals in these areas concur or not, these 
volunteer activities do take place and will 
continue to do so. Education, both formal, 
general, and vocational, becomes one of the 
mechanisms through which volunteers can 
learn appropriate S' & and roles and can 
therefore contribute to the well-being of the 
society. 

Establishing a scale of values for out- 
comes of this sort is not easy. This difficulty, 
however, is a very inadequate reason for 
ignoring the contribution made by the non- 
labor market elements of the society. The 
presence or absence of skills, and reports of 
engagement in nonpaid volunteer activities, 
may provide a gross estimate. Some attempts 
have been made to place dollar values on 
such services (e.g., the value of homemak- 
ing) but to a very large degree the problem is 
unresolved. Nevertheless, the contribution is 
too great to ignore. 

Closely related to the essential services 
is the outcome termed quality of work prod- 
ucts. It extends beyond the tangible and 
material quality of things to the quality of 
services provided. A disillusionment with 
quality in this broad sense leads to an ero- 
sion of stability and a reduction in the overall 
quality of life for the society as a whole. 



We are not aware of a body of literature 
that specifically documents this assertion, 
but several examples will illustrate the intui- 
tive logic of this position. First, consider the 
current plight of the automobile industry. 
Beginning in approximately 1974, the declin- 
ing demand for American automobiles 
became increasingly recognized. One result 
was that the market for replacement automo- 
biles began to decline as consumers chose to 
retain the vehicles they owned rather than 
pay higher prices for less satisfactory vehi- 
cles. This, in turn, reduced the effectiveness 
of pollution control devices because non- 
equipped automobiles stayed on the road 
longer. It also reduced the contribution of the 
auto industry to the economy and was a large 
factor in the turn toward imported autos. 
which further eroded the economy within the 
United States. Ten years later the conse- 
quences are still strongly felt. 

A second example applies to a service 
area. Suppose a child from a low socioeco- 
nomic status family is having serious achieve- 
ment trouble in school. The mother visits the 
child's teacher to see what can be done. Both 
the mother and the teacher are sincerely 
interested in the child's welfare. But the 
teacher does not know how to empathize and 
communicate with the mother, and the 
mother is made to feel inferior and inade- 
quate. She therefore avoids continuing 
school visits, and, as a result, the real contri- 
bution she could make is lost. The child's 
education continues to suffer. 

A third example, in an entirely different 
area, concerns a hypothetical situation in the 
delivery of medical care. A patient goes to a 
physician's office with an annoying but not 
too painful ailment, as yet undiagnosed. The 
patient is treated with perfunctory courtesy. 
More attention is given to the ability to pay 
than to diagnosis of the ailment, and the phy- 
sician appears to be hurried and not very 
interested. The next time the symptoms 
appear, the patient puts off seeking medical 
service until a serious and perhaps irreversi- 
ble condition develops. 
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We have emphasized in these examples 
the service products more than the tangible 
products. Deterioration in the quality of 
either contributes to the disruption of society 
through disillusion with its institutions, 
through displacement of people (witness the 
Michigan-Arirona migration phenomenon as 
displaced auto workers sought jobs in the 
Southwest), and through more extreme dis- 
ruptions such as hots, hijackings, and the 
organizing of private armies. 

These results are, of course, not attribut- 
able to lowered product quality alone, but 
such a phenomenon increases the potential 
for serious societal consequences 

The third outcome suggested in figure 1 
as relating to social stability is the accumula- 
tion of the necessary resources for self- 
support by the substantial majority of the 
members of a society. The proportion of the 
society engaged in productive work that may 
be exchanged for other goods and services is 
a fur' lion of both the standard of living and 
the productivity of the society. 

If individual productivity is relatively low 
and the desired standard of living is high, 
then proportionately more persons will need 
to be employed in a self-supporting way in 
order to carry the remaining members of the 
society at a similar level. Conversely, a 
smaller proportion of more highly productive 
self-supporting people can carry the remain- 
ing members assuming the standard of living 
stays constant. If, however, the standard of 
living is lower, less individual productivity is 
necessary to maintain the living standard. In 
immediate and practical terms, the actuarial 
difficulties of the Social Security system illus- 
trate this problem. As the proportion of 
retired workers who are no longer productive 
rises in relation to those paying Social 
Security taxes, the Nation is forced to choose 
some combination of four alternatives: 

• Accept a lower standard of living due 
to higher taxes and inflation 
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• Impose a lower standard of living on 
the retired group by maintaining a 
fixed income level despite inflation 

• Increase sharply the productivity of 
the working group 

• Encourage continued productivity of 
the retired group 

It is again difficult to specify just how 
progress toward this goal should be mea- 
sured. However, a reasonable goal to which 
vocational education should contribute is the 
increase or maintenance of such a productive 
majority. 

A society in which every individual con- 
tributes to his or her own betterment also 
contributes to the betterment of the lives of 
others in that society. This is what is meant 
by societal improvement— working as indi- 
viduals to improve the quality of life for the 
aggregate. This type of positive growth is 
considered necessary in order for a society 
to sustain itself rather than stagnate. Thus, 
societal improvement is the final critical ele- 
ment supporting social stability. 



Societal Outcomes Specific to Employers 

A subset of the societal outcomes are 
those that impact employers directly. To be 
worth the investment from the employer s 
point of View, vocational education should 
contribute to ensuring that needed skills are 
available, that on-the-job training time for 
graduates of vocational programs is less than 
for those without such training, and that the 
productivity of these trained workers is 
higher than that of untrained workers. To 
date, the evaluation of vocational education 
has been largely limited to factors such as 
these, while ignoring the numerous other 
outcomes suggested in figure 1 . It is our 
judgment that although these outcomes are 
necessary in the paid work market, they are 
not sufficient for assessing the quality of 
vocational education there or in the other 
defined outcome areas. 
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The Place of Vocational Education 

We began this discussion by considering 
the role of vocational education as a part of 
the broad concept of education in general. 
We have not been willing to accept a narrow 
definition that divorces the training in spe- 
cific, employment-related skills from the 
education of a person for.all aspects of living. 
Thus, our definition of vocational education 
includes, but is not limited to. employment- 
related skills. Development of those skills is a 
necessary, though not sufficient, component 
of vocational education as we see it. This 
leads to a recognition that vocational educa- 
tion of a positive quality must develop 
employment-related skills; it may not sub- 
stantially interfere with the development of 
desirable or even essential skills or disposi- 
tions that are not directly employment 
related; and it should, to the extent possible, 
enhance those dispositions. (For a succinct, 
normative discussion of some of these dispo- 
sitions, seeThimmesh 1984.) 



At the secondary level, vocational educa- 
tion competes for class time with other elec- 
tees and with required courses that reflect 
American society's collective judgment about 
what is basic and essential. At the postsec- 
ondary level, 0 similar competition is present 
in the planned sequences but is less enforced 
because of the greater legitimacy of taking 
one of two courses for immediate personal 
goals and then discontinuing the program * 
These conditions require that vocational 
education share responsibility for the kinds 
of goals shown in figure 1 , although it cannot 
be held responsible for their complete 
attainment, nor can the larger subset of for- 
mal education. It logically follows that the 
quality of vocational education can only be 
assessed by taking into account its role in the 
larger context 



•In addition. 9ome people assume that some of the goals have already been attained by the time of secondary school com 
pletion and consequently need not oe readdressed at the postsecondary level 
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CHAPTER 3 
THE EDUCATIONAL EXPERIENCE 



General Definition of the 
Educational Experience 

The vocational educational experience 
involves a network of interrelated and inter- 
dependent people, processes, and structures. 
A complex environment comprising a 
number of interacting parts, it functions 
within the general education system and 
within a much larger environmental context. 
In order to determine whether the environ- 
ment is operating effectively in achieving 
desired individual and societal outcomes and 
in order to make improvements in the educa- 
tional experience as a whole, it is essential to 
understand how each part functions, both 
independently and interdependently. Thus, 
defining in detail the elements of quality in 
the educational experience is a critical step 
toward achieving these ends. 

The conceptual framework proposed in 
figure 1 provides a starting point for discus- 
sion. The educational experience repre- 
sented by the central panel of the model is 
situated within the larger context of environ- 
mental inputs and outcomes shown in the 
left-hand and right-hand panels. The central 
panel is divided into "SChool Context," dis- 
played in the uppermost cell, and "School 
Process/ 1 represented by the three lower 
cells of the panel. The components listed 
within all four cells define the boundaries of 
the school context and school process 
domains. 



School Context 

School context deals with both organiza- 
tional and physical aspects of the school 
unit. It is represented in figure 1 as the first 
cell in the central panel. Personnel, facilities, 



equipment, policies, and formal organiza- 
tional structure are general elements of 
school context. 

School context sets the stage for school 
process and can best be described through 
illustration. The following two fictitious "con- 
text" scenarios help to show the place of 
school context within the overall educational 
experience. 

The development of Middletown Techni- 
cal Institute (MTI), a postsecondary voca- 
tional education school, was a direct 
outgrowth of current and projected needs of 
business and industry in Middletown, U.S.A. 
Goals of the school and its individual pro- 
grams were developed by a committee that 
included educators, administrators, students, 
parents, and representatives of business and 
industry. Consideration of school and pro- 
gram goals influenced the design of the phys- 
ical structure of the school as well as the 
purchase of equipment. Local business and 
industry demonstrated support of MTI 
through generous donations of equipment 
and material. Selection criteria for instructors 
included technical experience and knowl- 
edge, formal training in education methods, 
enthusiasm for teaching, and other selected 
variables. An active business-industry advi- 
sory board helped MTI stay abreast of chang- 
ing needs. Qualified counselors were 
charged with tailoring student programs to 
individual needs and assisting in job-seeking 
efforts. MTI enjoyed a favorable image in the 
community. There were more applicants to 
MTI than its programs could handle; student 
selection criteria were established to manage 
the demand overload. 

Steeltown Vocational School (SVS), our 
second fictitious program, presents a much 
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different picture of school context. SVS is a 
postsecondary institution that was opened in 
the early 19§0s to train skilled laborers for 
the local steel and machine tool industries. 
School and program goals have not been 
modified since its inception. The school 
building is a converted warehouse and the 
instructional staff is mainly composed of 
former supervisors of skilled laborers. 
Instructors have little or no formal education 
in teaching methods. The physical facility is 
in desperate need of repair having cracked 
walls and ceilings and antiquated plumbing, 
heating, and lighting systems. However, with 
enrollment and revenues down, there are no 
plans for making the, needed repairs. Place- 
ment rates for SVS graduates are at an all- 
time low. Support from the community is 
virtually nonexistent; it is only a matter of 
time before SVS will'be closed. 

These scenarios help to illustrate the 
importance of school context in "setting the 
stage" for the educational experience. Per- 
sonnel, facilities, equipment, and organiza- 
tional structure are all part of the "context" of 
an educational facility and relate to the qual- 
ity of a facility. The contrast betweenthe 
school context of MTJ^and SVS was exagger- 
ated in order to demonstrate the significance 
of context in the educational experience. A 
more detailed discussion of the components 
and elements of school context will appear 
later in this chapter. 

School Process 

If school context provides the setting, 
school process provides the action. Referring 
again to figure 1, school process is repre- 
sented by the second, third, and fourth cells 
of the central panel. Cell 2, entitled "Student 
Utilization, 1 ' relates to patterns ol participa- 
tion that are influenced by school context 
and school process (Campbell, Gardner, and 
Seitz 1982). Cell 3, "Teacher, Student, Mate- 
rial Interaction/' combined with Cell 4, 
"Instructional Content and Equipment; 1 
defines the teaching-learning process. 



Descriptions of school process include 
references to course content, course-specific 
equipment, and materials. Discussions of 
student-teacher, student-student, student- 
material interface, and level of formal student 
involvement in the vocational education 
school unit or program are also involved in a 
comprehensive description and analysis of 
school process qualfty. 

The following description of a fictitious, 
comprehensive high school may help to illus- 
trate the general boundaries of the school 
process domain. 

Jefferson Comprehensive High School is 
viewed mainly as a college preparatory insti- , 
tution. Students are discouraged from for- 
mally entering the vocational education 
program, so those students that take voca- 
tional education coursework do so mainly as 
a diversion from purely ac&demic course 
loads. Concentrators are generally held in 
low esteem by their peers and by the majority 
of academic program instructional staff. 
Vocational education instructors have little 
influence on purchase of instructional 
equipment and are forced to use out-dated 
textbooks and equipment. Students have 
limited opportunity for "hands-on" expe- 
rience. These restrictions and limitations are 
also reflected in episodes of student-teacher, 
student-student, and student-material inter- 
action. Instruction in vocational education 
courses appears to be a one-way process. 
Teachers dispense information and students 
absorb information; a feedback loop is virtu- 
ally nonexistent between teacher and stu- 
de«J* Typically, student-student interface 
consists of rubber band shooting, note pass- 
ing, and joke telling. Because of the lack of 
support and resources for vocational educa- 
tion programs, there is a high rate of turnover 
of 'good* vocational education teachers. It 
seems that those who stay are the type who 
care little for the job and the well-being of the 
student. 

This description touches upon the major 
components of quality in the school process 
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portion of the model. It provides a general 
"feel" for the school process domain and the 
potential effect of process on the overall 
educational experience. 



Interrelationship between the 
Educational Experience and Outcome 



The processes and structures that define 
the elements of quality within the vocational 
education experience should relate to 
intended individual and societal outcomes. 
The educational experience should increase 
the likelihood that desired individual and 
societal outcomes are achieved. In order to 
do so, a systematic understanding is required 
of the relationships between components of 
the educational experience and outcomes. 
This will allow the school unit to maintain 
and replicate effective programs and will 
promote systematic program modification as 
outcome inoicators change. 

For example, with the current emphasis 
on and movement toward electronic data 
processing both at home and on the job, 
computer literacy may eventually be included 
along with basic math, English, and so forth 
as part of basic skills. In turn, the addition of 
computer literacy should be reflected at the 
outcome indicator level and also in the form 
of school context and school process indica- 
tors. In other words, there should be a sys- 
tematic linkage between desired outcome 
and experience; the educational experience 
should be responsive to changes in desired 
outcomes. 

In order for the linkage to be systematic, 
the educational experience must be dis- 
sected into observable components or ele- 
ments of quality that relate to intermediate 
objectives. Achievement of these objectives 
should support the achievement of desired 
outcomes. Thus, outcomes are considered as 
goals, and the educational experience as the 
way to move closer to the goals. 



Elements of School Context 

A number of components related to qual- 
ity of education are classified within the 
school context and process domains of our 
model. For example, as figure 1 indicates, the 
components of school context are these: 

• School setting 

• Classroom setting 

• Teacher skills 

• Teacher characteristics 

• Peer group characteristics 

Each component consists of a set of ele- 
ments defining its parameters. A thorough 
understanding of elements of quality is pre- 
requisite to developing specific indicators of 
quality. Therefore, the elements of quality 
related to school context and school process 
will now be identified and discussed in detail. 



School Setting 

Our model proposes five components of 
quality within the school context domain (see 
figure 2). The first of these is school setting. 
School setting encompasses both physical 
and organizational aspects of the educational 
experience. The physical elements of school 
setting include the school building, the 
school property, and general equipment as 
opposed to program-specific equipment. 
Integrity of basic systems such as structural 
support systems, plumbing and heating sys- 
tems, ventilation and lighting systems, and 
exterior and interior design features would be 
considered as part of the "school building." 
The second element, school property, deals 
with the adequacy of adjunct facilities, such 
as parking lots. The category of general sup- 
port equipment is the third physical element 
related to quality of school setting. General 
support equipment is defined as equipment 
related to the functioning of the school unit 
as a whole and does not include equipment 
purchased for individual programs. Copy 
machines, telephones, file cabinets, and 
management related hardware and software 
are examples of general support equipment. 
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School Setting 



Classroom Setting 



Physical Aspects 

• school building 

• school property 

• general equipment 

Organizational 
Aspects 

• organization 

• strategy 

• policy 



Physical Aspects 
of Instructional 
Setting 

• systems 

• furnishings 

• design features 

• general 

instructional 
equipment 



SCHOOL 
CONTEXT 




Teacher Skills 



Technical Knowledge 

Technical Experience 

Teaching Methods, 
Knowledge, and 
Skills 



Teacher Characteristics 



Teaching Approach 
Teaching Style 



/ 




Peer Group Character iuticr 



Attitude toward Education 

Academic Performance 

Behaviors in Instructional 
Setting 
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Figure 2. Elements of quality of school context 
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A lathe is considered program-specific 

equipment and would not be included here. 
« 

Elements within the organizational 
aspect of school setting include formal 
organizational structure, strategies, and poli- 
cies. Organizational structure relates to the 
formal organizational chart and related man- 
agement philosophy.* It plays a major role in 
defining school setting since it directly 
affects personnel networks and information 
flow. For instance, one school unit might 
have a rigid, hierarchical organization chart 
with an autocratic management approach, a 
top-down flow of information, and a principal 
who is primarily concerned with manage- 
ment. A second school might also have a 
hierarchical structure but one that is defined 
by a participative management style (e.g., 
collegiality at the community college level), 
an interactive communication network, and 
an administrator who is primarily concerned 
with instruction. 

Strategies relate to long-range goals and 
directions of the school as a unit Research 
indicates that schools with clearly stated 
mission statements *are more effective (AASA 
1983). The basic strategic alternative of 
growth vs. no-growth is an.example of a stra- 
tegic direction that would affect school set- 
ting. An example of a strategic goal would be 
minimization of capital expenditures for 
equipment through development of and 
emphasis on a strong industry donation pro- 
gram. This goal could potentially affect the 
activities and responsibilities of administra- 
tors, instructors, and students and would 
therefore need to be consideredjna descrip- 
tion of school setting. 

Organizational policy is the final element 
within the organizational aspects of school 
setting. Policy affects the daily operation of 
the organizaton. Dress code is one example 
of a policy decision. Or, consider a 2-year 
college policy regarding credit for nontradi- 
tional learning experiences. 

Thorough consideration of the physical 
and organizational aspects of school setting 



•Administrator skills and characteristics are subsumed 



should result in a comprehensive description 
of the subelements of quality within that 
domain. Only then can judgments be made of 
adequacy, appropriateness, and effectiveness 
and indicators of quality be developed. 



Classroom Setting 

Classroom setting is the second compo- 
nent of school context. It is much narrower in 
focus than school setting. Classroom setting 
deals with physical aspects of the classroom 
as they relate to general instruction. Class- 
room setting deals with the overall physical 
comfort, safety, and adequacy of the general 
instructional setting (Westbrook 1982). 
Aspects to be considered would include ade- 
quacy of safety equipment: adequacy of light- 
ing, ventitation and heating; adequacy of 
desks and work areas; and availability of 
basic teaching equipment such as black- 
boards, pointers, flip charts, and more tech- 
nologically advanced teaching equipment 
such as videotapes, computers, closed circuit 
TVs. and so forth. Classroom setting is of 
particular importance in a vocational educa- 
tion setting. For example, in a course such as 
a welding lab where open flames are neces- 
sary, ventilation and traffic flow patterns are 
of extreme importance. Availability and ade- 
quacy of safety equipment and procedures 
are also critical. Elements of classroom set- 
ting are important to general instructional 
settings too. Aflost of us know what is it like to 
take a test in a cold, poorly lit classroom so 
we can understand why classroom setting 
has an effect on the overall educational 
experienc/8. The underlying premise is that 
certain standards should be met in order to 
enhanc/ learning and increase the probabil- 
ity of meeting course objectives. 



Teacher Skills 

The third component of school context is 
teacher skills. Three basic elements of quality 
are included in our definition of teacher 
skills. First is technical experience— How 
much direct work experience do teachers 

i the elements of the organizational aspect of school setting 
17 
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have in the fields in which they are teaching? 
Second is teaching methods, knowledge, and 
skills—Has the teacher completed postsec- 
ondary course work in history of education, 
teaching methods, approaches, and tech- 
niques? Does the teacher apply this knowl- 
edge to the classroom setting? Does the 
teacher present subject matter in a form or 
manner that makes it more meaningful and 
significant to the learner (National Commis- 
sion on Secondary Vocational Education 
1984)? Third is technical knowledge— Is the 
teacher technologically up-to-date (National 
Research Council 1983)? Has the teacher 
continued formal course work related to the 
teaching field? Does the teacher have recent 
field experience? These are the elements to 
be considered and some general questions 
addressed when assessing the teacher skills 
component of school context. 



Teacher Characteristics 

The fourth component of school context 
* is teacher characteristics, defined by two 
quality-related elements: teaching approach 
and teaching style. Teacher characteristics is 
a much "softer 11 component than teacher , 
skills, tyit it is no less important in terms of 
its effect on the educational experience. 

Teaching approach is related to teaching 
philosophy and is manifested as attitude 
toward the teaching-learning process as 
demonstrated through daily behavior both in 
and out of the classroom. Is learning a one- 
way process with the teacher dispensing 
information and the student absorbing infor- 
mation, or is the student-teacher feedback 
loop a critical element of process? Is learning 
to be dreaded or enjoyed? Does learning 
have a definite start and finish or is it a life- 
long process? Are the teacher and student on 
the same team or opposing teams? The 
answers to these types of questions help to 
define teaching approach. 

Teaching style is the result of a number 
of factors. Attitude toward job, attitude 
toward students, subject and content area, 



and individual personality all affect teaching 
style. Teaching style is expressed through 
teacher classroom behavior. Is the teacher 
enthusiastic and innovative? Is humor incor- 
porated, as appropriate, into the teaching 
process? Does the teacher show empathy 
with students? Is the teacher concerned with 
individual students? The teacher behaviors 
that provide the answers to these and similar 
types of questions constitute teaching style. 

In summary, the teacher characteristics 
category is the combination of a range of var- 
iables falling within the elements of style and 
approach. There is undoubtedly a range of 
effective teaching styles and approaches 
However, in order to develop indicators^ 
quality teaGher characteristics, the assump- 
tion must be made that there are identifiable 
characteristics that are common to effective 
styles and approaches. Most of us have 
memories of specific teachers whose styles 
and approaches were either particularly 
effective or ineffective. One can therefore 
easily understand why teacher characteris- 
tics play an important part of the overall 
learning experience. 



Peer Group Characteristics 

The fifth and final component of school 
context is peer group characteristics. This 
can generally be deftfWas the attitude 
toward education, acadamic performance, 
and behavior of the student body as a whole 
or of a homogeneous group within the stu- 
dent body. For example, secondary educa- 
tion could be synonymous with college 
preparatory education for one student body. 
Another student group may view high school 
as a setting for courtship and socialization, a 
third may view secondary education as a 
stepping stone to the world of work, or a 
fourth group might view it as f fitting for 
organized amateur athletic competition This 
aggregate view pervades the group and typi- 
cally affects students' behavior and perfor- 
mance at the individual level. Peer pressure 
plays a significant part in the formulation vid 
maintenance of peer group characteristics. 
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Consider the following example. A homo- 
genous group of very bright students decides 
that "grades" are meaningless symbo's. 
They claim that course content is noi chal- 
lenging and "anyone can get an A." There- 
fore, grades do not differentiate the bright 
students from the marginal students. This 
peer group holds the learning process and 
school programs in very low esteem. They 
decide to express this attitude through poor 
attendance records, nonparticipatory behav- 
ior in the classroom, and failing grades in all 
their courses. In doing so, they are making a 
group statement about the educational expe- 
rience that affects group members* academic 
performance and behavior at the individual 
level. 

Thus, peer group characteristics may 
significantly affect the student-learning inter- 
face and can be perpetuated or modified 
through peer pressure systems. An under- 
standing of peer group characteristics is pre- 
requisite to a comprehensive definition of 
school context. 

School context "sets the stage" for 
school process, which will provide the 
"action." Context relates to the quality of the 
physical and organizational setting ^through 
the school setting and classroom setting 
components and provides characterization of 
the main actors through the components of 
teacher skills, teacher characteristics, peer 
group characteristics, and school setting 
since it includes administrative skills and 
characteristics. 



Elements of School Process 

School process is the remaining portion 
of our conceptualization of elements of qual- 
ity in the educational experience. The 
teaching-learning process and patterns of 
participation are its major subdivisions. 
Components and elements were identified 
and defined as part of the model formulation 
process described in chapter 1. The results of 
that effort as they relate to school process 
follow. 



Teaching-Learning Process 

The teaching-learning process includes 
two components related to quality: teacher 
student, and material interaction and instruc- 
tional content and equipment. These compo- 
nents are represented as cells three and four 
of the central panel in our conceptual frame- . 
work (figure 1) and are further detailed in 
figure 3. 

Instructional content and equipment. 

Instructional content and equipment deals 
with course-specific information and equip- 
ment. Even though there is a wide range of 
course offerings within any given school unit, 
there are certain common elements related to 
quality of content and equipment that should 
be considered regardless of the course. 
These elements include but are not limited to 
the following: 

• Clear course objectives (Westbrook 
1982) 

• Business- and industry-related 
course content (National Research 
Council 1983; Benson 1982) 

• Instructional content related to 
course objectives (Westbrook 1982: 
Miller 1984) 

• Course objectives consistent with 
program goals (Westbrook 1982; 
Miller 1984) 

• Appropriate equipment available 

• Utilization of multiple media for 
instruction 

• Balance between materials requiring 
content knowledge, understanding, 
and application (Benson 1982) 

• Balance between knowledge, skill, 
and attitude development (National 
Research Council 1983) 

These elements and others like them 
serve as a starting point for the development 
of indicators of quality content and equip- 
ment. The underlying premise is that certain 
standards indicative of quality can be identi- 
fied for each element. 
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Teaching- Learning Process 



Instructional 
Content and 
Equipment 



Teacher, Student, 
Material Interaction 



• Clear Course Objectives 

• Business- and Industry- 

related Course Content 

• Instructional Content 

Related to Course 
Objectives 

• Course Objectives 
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Program Goals 

• ApproprjalSTifauipment 

Available 

• Use of Multimedia 

for Instruction 

• Balance between Materials 

Requiring Content 
.Knowledge, Understanding, 
|and Application 
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Skill, and Attitude Development 



? Type 

• Class 

• Frequency 

• Duration 

• Nature 



Student Utilization of 
Vocational Education 



Patterns of Participation 

• Concentrators 

• Limited Concentrators 

• Concentrator/Explorers 

• Explorers 

• Incidental/Personals 



Figure 3. Elements of quality of school process 
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Teacher, student, material interaction. 

The second component of the teaching/ 
learning process is labeled teacher, student 
material interaction. This component can 
generally be described in terms of five de- 
scriptive elements: type, class, frequency, 
duratl&n, and nature 4 



• Type refers to the specific, interactive 
combination (e.g., student with stu- 
dent, student with teacher, etc.). 

• Class refers to the basic classification 
of the interaction. Examples 
include— 

—teacher monitors student, 
—teacher questions student, 
—teacher reinforces student, 
—student reads material, 
—student handles equipment, 
—student questions teacher. 

• Frequency involves an actual count 
of interactions by type or class. For 
instance, the number of student-with- 
teacher interactions occurring within 
a class period might be noted. 

• Duration involves a time measure of 
the length of each interaction. This is 
usually recorded by type of interac- 
tion (e.g., How long did the student 
with student interface last?). 

• Nature involves a general assessment 
of the quality of the interface, such as 
positive, neutral, or negative. 

A considerable number of references 
aided in the development of this classifica- 
tion scheme. For example, Halasz, Behm, and 
Fisch (1984), Miller (1984), and Westbrook 
(1982) suggested an increase of "time on 
task" in effective programs. This would 
involve development of indicators that 
include type, class, and duration elements. 
Hunt and Woods (1979) recommended fre- 
quent positive reinforcement of student by 
teacher that would include the elements of 
type, class, and nature. Westbrook (1982) 



and Miller (1984) suggested that teachers 
should monitor student performance and 
provide qualitative feedback. This sort of 
interaction would be described by a combi- 
nation of all five elements in our scheme. 
Therefore, references helped to identify ele- 
ments related to quality of thefeeefier, eftK - 
dent, material interaction component. 

Thus, a description of student teacher, 
and material interaction in the classroom 
would include a statement of type, class, fre- 
quency, duration, and nature. Development 
of indicators of quality for these interactions 
would then be made in consideration of 
instructional content and equipment. For 
instance, it may be determined that a lab set- 
ting requires a greater frequency and dura- 
tion of student-with-student interaction than 
does a lecture class. Or, there may be a 
higher need for teachers to monitor individ- 
ual students in a lab setting than in a class- 
room setting. 

Therefore, a comprehensive analysis of 
the quality of the teaching-learning process 
component would include references to both 
instructional content and equipment and 
teacher, student material interaction. This 
process provides much of the "action" in the 
school unit and is clearly a major part of the 
overall quality of the educational experience. 

Student utilization of instructional pro- 
cess. The balance of school process deals 
with the formal relationship between the stu- 
dent and the vocational education school 
unit or program. Campbell etal. (1982) iden- 
tified this interface as patterns ofparticipt 
tion\ it has been incorporated into our model 
as a component of school process (figure 3). 
but it is also accurately described as an 
intermediate outcome of the interaction 
between school context and school process 
(figure 1). v 

Campbell conceptualized a five- 
dimensional matrix for describing possible 
patterns of participation. The dimensions of 
the matrix were intensity, diversity, conti- 
nuity, supportive diversity, and proximity. 
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Five patterns were hypothesized and empiri- 
cally yerified. These patterns were identified 
by Ompbell, Gardner and Seitz (1982) as 
follows: 

• Concentrators 

. • Limited ^Concentrators 

• Concentrator/Explorers 

• Explorers 

• Incidental/Personals 

Briefly, the descriptive concepts include: 
(1) the number of credits received in 
vocational courses in the program area 
of specialization; (2> the number of pro- 
gram areas in which the specialty was 
pursued; (3) the number of vocational 
credits in the program area that were 
determined to be supportive of the spe- 
cialty area; (4) a scaled measure of 
whether the specialty was pursued in the 
11th or 12th grade; and (5) the number of 
years in which the specialty was taken. A 
student s area of specialization was 
defined as a program area (for example, 
distributive education, home economics) 
in which at least six-tenths of the tqfal 
number of vocational credits were 
received. 

For example, Concentrators take an 
average of 6 vocational credits over a 3- 
year period. Limited Concentrators 
generally take about half the number of 
vocational credits that Concentrators 
take, usually within a 2-year period. 
Concentrator/Explorer is similar to the 
Limited Concentrator pattern, except 
that the vocational course work is usually 
completed early in the high school years. 



Students classified in the Explorer pat- 
terns pursue courses in three or more 
program areas but do not achieve any 
level of specialization. In comparison, 
Incidental/Personal students average 
less than a full credit and generally com- 
plete the work in a semester. These pat- 
terns were used in the analyses in place 
of the traditional curriculum descriptors 
of vocational, general, and college 
preparatory, (p. 16) 

Although detailed discussion of these 
patterns goes beyond the scope of this 
report, it is important to be aware that the 
extent and kind of participation in vocational 
education is critical to a comprehensive eval- 
uation of quality of the educational expe- 
rience. Patterns of participation seem to be , 
influenced by a number of aspects of the 
educational experience, such as program 
structure, use of facilities and equipment, 
assignments of students, course objectives, 
and elements of teacher, student, and mate- 
rial interface. In other words, patterns of 
participation are interrelated with the 
teaching-learning process arid school con- 
text and may therefore be described as an 
intermediate outcome of their interaction. 

Our conceptualization of elements of 
quality in the educational experience pro- 
vides a framework for developing indicators 
of quality. The model may not be compre- 
hensive, but a continuation of the process by 
which it was developed should lead to a 
complete definition of the concept. A general 
discussion regarding development of indica- 
tors of quality will be presented in chapter 4. 
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CHAPTER 4 



ASSESSING QUALITY OF VOCATIONAL EDUCATION 



Introduction 

It is no longer sufficient to say Johnny 
can read. One must also state how well he 
can read, why he can read, and what can be 
done to improve Johnny's reading ability. 
Consequently, evaluation has become a 
major activity within the field of education 
today. The individual, Federal, State, and 
local agencies and society as a whole have 
concerns about the quality of educational 
processes, structures, and outcomes. There- 
fore, educators are being asked to assess a 
complex educational system by means of a 
complex evaluation system that is quite a 
complex undertaking in itself. 

Evaluation is a major area of emphasis in 
the Carl D. Perkins Vocational Education Act 
of 1984. In addition to the requirement for 
placement and employment data for pro- 
grams assisted under the act, each State is 
required to evaluate program effectiveness 
and "quality and availability of instructional 
offerings." Consequently, within vocational 
education today, the evaluation of quality is 
emphasized through its ties to legislation and 
funding. 

Chapter 4 presents some general, yet 
important, issues and guidelines related to 
assessing quality of vocational education. It 
is not intended to be a comprehensive review 
of program evaluation techniques, models, 
and methods; the literature on the topic is 
voluminous and could only be superficially 
treated within the scope of this writing. 
Rather, general evaluation approaches, spe- 
cific models, and so forth are discussed as 
they relate to developing indicators for the 
elements of quality vocational education that 
were discussed in chapters 2 and 3. 



What It Quality? 

Chapters 1, 2, and 3 presented within a 
systems framework the components and 
elements that affect quality of vocational 
education. These components and elements 
are generalizable across school units and 
programs. But the specific indicators of qual- 
ity associated with them may vary with 
school unit and program and are likely to 
change over time. This is because specific 
definitions of what indicates quality are rela- 
tive, judgmental, and subject to change as 
the scope of the individual provider respon- 
sibility changes. 

The definition of quality education is 
strongly influenced by conceptual 
approaches operant within the field at a par- 
ticular point in time/Therefore, the definition 
of "good" depends on present knowledge 
and opinion. For example, in the early 1970s, 
the "open" classroom approach was adopted 
by many educators. Programs were modified 
and facilities were tailored to this new 
approach with the underlying assumption 
that the "open" classroom was positively 
related to "quality" education. 

Consequently, although the components 
and elements of school context, school pro- 
cess, and particularly the educational out- 
comes remained intact, specific standards of 
quality related to those components probably 
changed. Consider, for example, classroom 
setting. One indicator of a quality classroom 
setting prior to the open classroom approach 
might have been this: 

• A closed, well-sound-proofed instruc- 
tional setting allowing X square feet 
per student and one desk per 
student. 
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For those schools that subsequently 
accepted the open classroom concept the 
parallel indicator would probably have been 

this: 

• An open instructional setting with no 
structural barriers between four 
independent classes and 'work tables 1 
designed for simultaneous use by 
four to six students (no individual 
desks for students or teachers). 

Of course, this is just one example of how a 
specific indicator of quality might change as 
approaches within the field change. In actual- 
ity, the open classroom approach undoubt- 
edly affected indicators associated with most 
of the elements o? the school context and 
school process portion of our framework. 

The definition of quality also changes as 
scope of provider responsibility changes. 
Consider the growing demand for and role 
of day-care services today. Many children 
from tha time they are infants are placed with 
day-care providers for 45 hours per week. 
These providers spend considerably more 
waking hours with these children than do 
their parents. The scope of the providers 1 role 
has potentially expanded from one of basic 
care provider to that of primary role model. 
The overall societal and individual assess- 
ment of the importance or substance of child 
raising has not changed. However, the defini- 
tion of day-care has expanded and the 
indicators of quality day-care should theoret- 
ically reflect this change. Specific indicators 
of quality are expected to change even 
though the basic conceptual framework for 
elements of quality education will remain the 
same. 

Consequently, the nature of quality 
makes assessment difficult but not impossi- 
ble. The difficulty in no way reduces the 
necessity. Popham and Grasso (cited in 
Grasso 1979) have stated that assessing the 
worth or merit of educational phenomena is 
an essential component of systematic educa- 
tional evaluation. In addition, section 112 
(b)(1 )(B) of the 1976 Amendments to the 



Vocational Education Act of 1963 as inter- 
preted by Federal regulations explicitly states 
the need for evaluating "quality and availabil- 
ity of instructional offerings" (Fdderal Regis- 
ter. 3 October 1977; section 104.402). Thus, 
the assessment of quality in vocational edu- 
cation is not only generally recognized and 
accepted as important, but it is mandated by 
law. 



Approaches to Attesting Quality 

Considerable effort has been expended 
in developing evaluative strategies to facili- 
tate the assessment of quality. Much of this 
work has been drawn from the areas of 
organizational behavior and systems theory 
and adapted to specific field evaluation mod- 
els. Three predominant approaches have 
emerged that are relevant to assessments of 
quality within the field of vocational educa- 
tion. We will generally refer to them as struc- 
tural, process, and outcome approaches. 



Structural Approaches 

Structural approaches focus on aspects 
of the organization that impact on quality of 
education. This would include such variables 
as personnel, facilities, equipment, formal 
organizational structure, policies, and so 
forth. These variables primarily relate to 
components and elements of quality within 
the school context domain of our model (fig- 
ures 1 and 2). For example, in a structural 
approach to4he issue of human resource 
development, a judgment might be made that 
certain teacher qualifications are necessary 
to ensure quality education. These qualifica- 
tions would be expressed as indicators within 
perhaps the teacher skills and teacher char- 
acteristics components of school context. Or, 
in terms of classroom setting, indicators 
might be developed that Specify teacher- 
student ratios and student-square feet ratios, 
and so forth. These would serve as standards 
of quality for instructional setting. 
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Two major assumptions support a struc- 
tural approach to assessing quality. First, it (s 
possible to identify "good" in terms of staff, 
physical facilities, formal organization, and 
so forth. And second, the quality of educa- 
tion will improve as a result of such things as 
better qualified staff, improved physical facili- 
ties and equipment, and sounder organiza- 
tional structure. 



Process Approaches 

Process approaches utilize criteria- 
oriented methods and individual studies to 
examine activities involving teachers, stu- 
dents, materials and their interaction, and 
other elements that are included within our 
definition of school process. Therefore, pro- 
cess approaches would primarily focus on 
the school process portion of our model (fig- 
ure 3). A process approach might examine 
such things as "hands-on" equipment time in 
a welding class, frequency of "teacher ques- 
tioning student" during a lecture class, or 
clarity of course objectives against selected 
indicators of quality. 

Process approaches to the assessment of 
quality assume that it is possible— 

1. to set intermediate course of program 
goals and evaluate whether goals 
have been met, 

OR 

2. to establish quantitative and qualita- 
tive standards of quality, based on 
expert opinion and applied to indi- 
vidual case review, 

OR 

S. to directly observe critical behaviors 
that permit determination of quality, 
AND 

4. for school educators and administra- 
tors to agree upon what constitutes 
quality without continually monitor- 
ing educational outcome. 

Process evaluations can be utilized to detect 
problems in program design and implementa- 
tion, to provide information for formative 



decision making, and to develop databases 
for long-range planning (Stufflebeam et al. 
cited in Grasso 1979). 



Outcome Approaches 

■j 

Outcome approaches to assessing qual- 
ity of education assume that certain direct 
measures of results are related to quality of 
education. Theoretically, within a pure 
outcome framework, the qualifications of 
educators, the structure of the organization, 
facilities and equipment, and school process 
components and their interactions can be 
disregarded in favor of assessing whether or 
not desired results are achieved. One exam- 
ple of an outcome approach is Scriven's con- 
ceptualization of payoff evaluation that 
appears to include analyses of both outcome 
and impact (cited in Grasso 1979). 

The underlying assumptions of an out- 
come approach are the following: 

1. There is a high level of agreement 
between societal, individual, and pro- 
fessional estimations as to what end 
results are desirable. 

2. Desired outcomes can be expressed 
as quantitative and qualitative indices 
of success. 

3. Desired outcomes are brought about 
to a great extent by quality 
education. 

4. There is a high level of agreement 
between experts and practitioners 
regarding validity of selected out- 
come indicators. 

An outcome approach would focus primarily 
on the elements of quality vocational educa- 
tion outcomes depicted in the right hand 
panel of our model (figure 1). Outcome indi- 
cators or sets of outcome indicators would 
be developed for each component in this 
outcome panel and a determination of 
attainment or lack of attainment would 
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subsequently be made. In some cases, the 
assessment could be immediate; in others, 
there may be a considerable time lag. For 
example, reading ability might be assessed 
by means of a reading achievement test bat- 
tery just prior to graduation. In some profes- 
sions, licensing exams are taken shortly after 
graduation and serve as indicators of attain- . 
ment of job-specific skills. But in the case of 
positive interpersonal interactions, a consid- 
erable period of time on the job may be 
required prior to assessment and the method 
of measurement may not be so readily 
available. 



A major difficulty with a pure outcome 
approach is that it requires no systematic link- 
age between desired outcomes and com- 
ponents and elements of the educational 
experience. Program modifications resulting 
from a pure outcome approach may not be 
systematic. The problem is due to the limited 
scope of the analysis. The same problem 
exists for pure structural and process 
approaches. 

An evaluation system designed to assess 
quality of vocational education as defined by 
our conceptual framework must consider 
school context, school process, and out- 
comes and therefore requires a synthesis of 
these approaches. An integrated approach to 
the evaluation of quality assumes that 
appropriate Structure (school context) 
increases the probability of quality programs 
(school process), which, in turn, increases 
the likelihood of successful outcome. 

The CIPP (Context, Input, Process, Prod- 
uct) model designed for institutional decision 
making (Stufflebeam f t al. cited in Grasso 
1979) is one example pta integrated evalua- 
tion approach that could be applied to the 
assessment of quality. Anothe would be 
Drewes' expansion of the CIPP model to 
include impact evaluation (Drewes et al. 
1975). Drewes 1 comprehensive model incor- 
porates structural, process, and outcome 
approaches within the following evaluation 
methodologies: 



• Context evaluation— identifies unmet 
needs and unused opportunities 

• Input evaluation— identifies and 
assesses merits of various structural 
variables such as staff, organization, 
and physical facilities 

• Process evaluation— reviews opera- 
tions of programs and the teaching- 
learning process as a whole 

• Product evaluation— measures and 
interprets attainments at end of 
courses and programs to assess 
intermediate goal attainment 

• Impact evaluation— examines 
intended and unintended results of 
vocational education, such as 
employee satisfaction and post- 
program placement 

• Palrwite evaluation— examines pair- 
wise interrelationships of evaluation 
types listed previously 

The primary point is that a comprehensive 
and systematic evaluation of the elements of 
vocational education quality requires a sys- 
tem of evaluation that includes structural, 
process, and outcome approaches. 



Developing Indicators of Quality 

Indicators are standards against which 
something, in this case, quality, can be mea- 
sured. In order to assess quality of vocational 
education,comprehensively, sets of indica- 
tors need to be developed for the elements of 
quality presented in chapter 2 (outcomes) 
and chapter 3 (educational experience) and 
incorporated into an evaluation system such 
as Drewes' model. Assessing quality within 
these constraints imposes formidable difficul- 
ties in terms of time and cost. It is not practi- 
cal to tabulate exhaustively all of the 
indicators related to the elements of our 
model, nor is it financially practical to 
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support an evaluation system capable of col- 
lecting and analyzing such data in a com* 
plete form. 

Our approach to this difficulty involves 
two initial steps: , 

. Prepare a, representative list of sub- 
elements. These should be selected 
to ercompass the range of each 
major component and element com- 
bination of our model as described in 
chapters 2 and 3. 

2. Secure a consensual judgment about 
; the relative priority of time or 

resource investment in the subele- 
ments. These priorities might serve 
as proxies for the overall value of the 
subelements. (AI[ow the essentiality 
of specific subelements to influence 
this priority. Some may be essential 
as preconditions or in their own right; 
others may be alternatives*) This 
exercise will also provide the oppor- 
tunity to suggest new subelements or 
to eliminate inappropriate ones. 

These steps should lead to the prioritization 
of elements within our framework. This order- 
ing may vary from school unit to school unit 
and within school units over time. But after 
these two steps are reasonably complete, it 
will be possible to develop a set of indicators 
for at least the high-priority subelements 
within the outcome and educational expe- 
rience portions of our model. 

A set of underlying assumptions is 
needed to support the development of quality 
indicators. The following assumptions have 
been extrapolated from Zusman and 
Lawson's (cited in Riedel et al. 1974) work 
related to assessing the quality of mental 
health patient care: 

• Indicators of "quality" education can 
be characterized and rated. 

• Definition of quality education as 
expressed through indicators 



depends on present knowledge and 
opinion. 

• Definition of quality education as 
expressed through indicators 
changes as scope-of-provider 
responsibility changes. 

• Decisions regarding adequacy of par- 
ticular educational structures and 
appropriateness of particular pro- 
cesses and programs are influenced 
by decision makers' value systems 
and judgments and are therefore 
ultimately subjective 

These assumptions take into account the rel- 
ative, judgmental, and changing nature of 
definitions of quality as expressed at the 
indicator level. For the purposes of the ensu- 
ing discussion, these assumptions will be 
considered valid. 

Normative versus Empirical 

Both normative and empirical 
approaches should be drawn upon for devel- 
oping indicators. Normative indicators would 
be formulated on the basis of an ideal of what 
represents excellence. Methods used to 
develop normative Criteria include judgments 
of experts or highly qualified panels and 
state-oMhe-art literature and research. Nor- 
mative standards are based more on a body 
of knowledge than actual practice. One prob- 
lem with a purely normative approach is that 
practitioners and educators feel that it is not 
practical front* a time, cost, and other per- 
spective to apply idealized standards to 
actual practice. Empirical standards, on the 
other hand, are based on actual practice- 
educational programs and courses. A major 
criticism of purely empirical approaches is 
that standards may appear adequate relative 
to other situations (school units, programs, 
courses) yet fall far short of what is achiev- 
able through application of current knowl- 
edge. A blend of the normative with the 
empirical is the recommended approach to 
developing indicators of quality. Empirical 
observation should provide a baseline that is 
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then modified by normative judgment; school 
units and programs should not lose sight of 
the ideal. 

Quantitative versus Qualitative 

There exists a long-standing debate con- 
cerning the advantages of qualitative versus 
quantitative methods of evaluation. However, 
a growing body of evaluation researchers 
holds that the two methods can be utilized 
quite profitably together. Each method has 
its biases, so selection of the method should 
be based on purpose of the evaluation and 
specific constraints connnected with the 
project. 

There are a number of reasons why a 
combination of qualitative and quantitative 
methods is suited to a comprehensive evalu- 
ation of quality. First, a comprehensive 
evaluation requires analyses of quality of 
structural, process, and outcome elements. 
This variety of analyses typically requires a 
variety of methods. Second, when used 
together for the same purpose, the two- 
method-type indicators can support one 
another and often lead to insights that 
neither one could provide independently. 
And third, utilizing a combination of qualita- 
tive and quantitative methods helps to 
account for the biases that are associated 
with each approach (Cook and Reichart 
1979). Evaluators need to fit the method and 
the indicators to the demands and purposes 
of the particular evaluation; restricting one- 
self to only one of the method types will 
probably restrict a comprehensive evaluation 
of quality. 



Examples of Indicators 

As previously noted, specific indicators 
of quality are typically not generalizable 
across school units or even school programs, 
so it is not practically possible to tabulate all 
potential indicators of the elements of quality 
in our model exhaustively. Therefore, just a 
few examples will be mentioned in order to 
clarify the relationship between indicators 



and model components and elements and to 
demonstrate the level of specificity involved 
at the indicator level These examples are 
shown in figure 4. 

Of course, these are only a few examples 
of indicators that might be associated with 
components and elements of quality voca- 
tional education. The key point is that indica- 
tors must be measurable or observable and 
considered as good proxies for the context, 
process, or outcome elements that they are 
intended to measure. It is realized that con- 
siderable effort is involved in developing 
indicators of quality. However, if the problem 
is attacked in a systematic and phased 
manner, it is possible for individual school 
units and programs to establish some valu- 
able indicators of quality. 



Summary 

The elements of quality proposed in this 
paper have been derived from a diverse body 
of literature and experience. We have attempt- 
ed to incorporate various findings related to 
attributes of quality education and programs 
into a general yet comprehensive conceptual 
framework in order to broaden our under- 
standing of the elements of quality vocational 
education. We hope it will contribute to an 
increased awareness of the major compo- 
nents influencing the quality of education. 
This report is intended to Inform policy- 
makers of the breadth of the elements of 
quality and to suggest an integrated frame- 
work for assessing quality to policymakers 
and vocational education evaluation 
theoreticians'. 

For purposes of this inquiry, vocational 
education was defined as school-related 
activities that occur in secondary schools 
and 1-and 2-year programs in postsecondary 
schools; vocational education was consid- 
ered to be a subset of general, formal educa- 
tion, which in turn is a subset of education 
coming from a variety of life experiences. 



28 



o 

ERIC 



38 



COMPONENT 



ELEMENTS 



INDICATOR 



Teacher Skills 



• Technical Knowledge 

• Technical Experience 



* Teaching Methods 



• Credentials, licenses 

• Teacher must have minimum of 

3 yea's of full-time work experi- 
ence in field prior to hire 

• Teacher must accumulate X num- 
ber of hours of field experience 
through co-op program or summer 
employment for each 5-year teach- 
ing period 

• Prior to hire, teacher must have 
20 quarter hours in teaching 
methods and 10 quarter hours 

in history-of-education course work 
from an accredited college 

• Teacher must take minimum of * 

4 quarter hours in teaching meth- 
ods course work from an accredited 
college for each 5 yeapof teaching 



Competency Development • Job-specific Skills 



• Program graduate receives a passing 
grade on licensing exam 



Student Utilization of 
Instructional Process 



• Patterns of 
Participation 



A minimum of 65% of vocational 
program graduates are classified as 
Concentrators and Limited Concen- 
trators 



Basic Skills 



• Reading 



Program graduate score ranks above 
50% percentile on standardized 
reading comprehension test 



Teacher, Student, and 
Material Interaction 



• Student, Material 



• Over a period of 4 lecture classes, 
student should spend no more 
than 25% of class time reading 
material in class 

• In metal-working lab, 2-person 
student teams should be working 
together between 40% and 60% 
of class time 



Figure 4. Examples of quality indicators 
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Our framework for the elements of qual- 
ity grew out of a multiphased process that 
included reviews of related literature, expert 
input, and key informant interviews. The 
literature provided preliminary structure to 
the framework which was further refined and 
defined by expert opinion and key informant 
commentaries. 

Some vocational education has been 
directed by law to be evaluated in terms that 
include job placement in trainirg-related 
employment, employer satisfaction, and qual- 
ity and availability of instructional offerings. 
This project, however, initially explored* va- 
riety of desirable outcomes that may help to 
define"high quality vocational education pro- 
grams and subsequently identified the com- 
ponents of the educational environment that 
may support attainment of those outcomes. 
The outcomes were therefore considered as 
goals and the structures and processes 
within the educational environment were 
perceived as means for achieving the goals. 
The resulting framework for the elements of 
vocational education quality (figure 1) was 
conceptualized as a complex environment 
converging on the educational experience* 
driven by desired individual and societal out- 
comes, and influenced by environmental 
inputs such as community setting and labor 
market conditions. 

, The Framework: Elements of Quality 
Vocational Education 



Elements of Quality Outcome 

Two general types of outcomes are 
included in our framework: individual and 
societal with a subset of societal outcomes 
specific to employers. Four elements of qual- 
ity relating to individual outcomes were iden- 
tified and defined. Competency development 
encompasses the development of work- 
related skills (both occupational and job spe- 
cific), basic skills such as verbal and 
mathematical, and general problem " 8 ng 
ability. Positive interpersonal interact v> 



involves the ability of people to have positive 
and mutually supportive relations with others 
in the home, the workplace, and the com- 
munity. A sense of efficacy is an attitude of 
confidence in one's ability.to utilize skills to 
achieve goals. The ability to manage a large 
array of tasks without having to depend upon 
someone else is designated as a sense of 
independence. These are the four general 
elements of quality individual outcomes 
included within our conceptual framework. 

5oc/a/ stability and employer needs are 
the primary elements of quality within the 
societal outcome portion of our framework. 
.Social stability does not imply social stagna- 
tion; rather, it means gradual, orderly ^ 
change. Provision of essential services, quel- 
ity of work products, resources for self- 
support, and societal improvement are the 
subelements that support social stability. 

ft 

We also examined a subset of societal 
outcomes specific to employers. To be worth 
the investment from the employees point of 
view, vocational education should contribute 
to available skills, reduced training time, and 
high productivity. These are the three ele- 
ments of quality related to employers needs. 
Our definition of quality vocational education 
outcomes includes, but is not limited to, 
employment-related skills. Rather, vocational 
education of a positive quality must also 
enhance the development of desirable or « 
even essential skills or dispositions that are 
not directly employment related. 

Elements of Quality within 
the Educational Experience 

Taking a systems perspective, the 
processes and structures that define the ele- 
ments of quality within the vocational educa- 
tion experience should relate to intended 
individual and societal outcomes. Our model 
divides the educational experience into two 
domains, school context and school process. 
School context deals with both physical and 
organizational aspects of trie school unit. 
Personnel, facilities, equipment, policies, and 
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formal organizational structure all fall within 
school context. The elements of quality 
w'thin school context that are formally 
included in our model are school setting, 
classrcom setting, teacher skills, teacher 
characteristics, and peer group characteris- 
tics. Figuratively speaking, school context 
provides the setting for school process, 
which provides the "action." Estimations of 
quality of school process would consider 
such things as course content; course- 
specific equipment; interaction between 
teacher, student and materials; and level of 
formal student involvement in the vocational 
education program. The elements of quality 
formally included within school process are 
teacher, student, and material interaction: 
instructional content and material', and stu- 
dent utilization, of instructional process. 

These elements of quality within the 
educational experience should relate to 
intermediate objectives that, in turn, should 
support attainment of desired outcomes; the 
linkage should be systematic hstween quality 
^of experience and quality of outcome in 
order to maintain and replicate effective pro- 
grams and make modifications as necessary. 



Assessing the Quality of 
Vocational Education 

The elements of quality within our 
framework are generalizable across school 
unit and program. However, the specific indi- 
cators of quality associated with them may 
vary across school and program and are 
likely to vary over time. 



Considerable effort has been expended 
in developing evaluative strategies to facili- 
tate the assessment of quality. Evaluation 
models should combine structural, process, 
and outcome approaches in order to assess 
the range of elements of quality within our 
framework. 

Specific indicators of quality need to be 
developed at the individual school unit level. 
It is not practically possible to tabulate 
exhaustively all the indicators related to the 
elements of our model, nor is it financially 
practical to support an evaluation system 
capable of gathering and analyzing such data 
in a complete form. A process for identifying 
and prioritizing subelements within the 
model is suggested as a practical starting 
point for developing indicators. A combina- 
tion of quantitative and qualitative methods is 
suited to a comprehensive evaluation of qual- 
ity, and both normative and empirical 
approaches should be drawn upon for devel- 
oping indicators. 

In summary, this discussion presents a 
justification and set of procedures that will 
permit an integrated consideration of the 
elements of vocational education quality. We 
have advocated considering a broad set of 
elements of quality rather than a narrowly 
defined or segregated set, so that the'evalua- 
tion of vocational education reflects its con- 
tribution to a set of living experiences in 
addition to employment. 
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